Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repeat option for e2e tests and incrementing timeout #6655

Merged

Conversation

niklasberglund
Copy link
Collaborator

@niklasberglund niklasberglund commented Aug 21, 2024

This PR introduces an option for specifying how many times e2e tests should run on our office runner(s), similar to the option mockapi already have. The timeout for mockapi and e2e tests is now also based on the number of test runs to avoid a high number of runs resulting in reaching timeout.

The changes can be tested by running the Android - Build and test workflow on this branch.

image

This change is Reviewable

@niklasberglund niklasberglund added the Android Issues related to Android label Aug 21, 2024
Copy link

linear bot commented Aug 21, 2024

@niklasberglund niklasberglund force-pushed the add-repeat-arguments-for-self-hosted-e2e-tests-droid-1266 branch from cbdfea2 to 43a8cdb Compare August 21, 2024 15:15
Copy link
Collaborator

@albin-mullvad albin-mullvad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @niklasberglund)


.github/workflows/android-app.yml line 457 at r2 (raw file):

      - name: Run instrumented test script
        if: ${{ matrix.test-repeat != 0 }}
        timeout-minutes: ${{ fromJSON(steps.calculate-timeout.outputs.timeout) }}

Do we need to parse these timeouts from json?

Code quote:

fromJSON

Copy link
Collaborator Author

@niklasberglund niklasberglund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @albin-mullvad)


.github/workflows/android-app.yml line 457 at r2 (raw file):
It does look very odd, but fromJSON can be used to convert string to number. And the value for timeout-minutes cannot be a numerical string.

"For numerical comparison, the fromJSON() function can be used to convert a string to a number"

https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions

Copy link
Collaborator

@albin-mullvad albin-mullvad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved


.github/workflows/android-app.yml line 457 at r2 (raw file):

Previously, niklasberglund (Niklas Berglund) wrote…

It does look very odd, but fromJSON can be used to convert string to number. And the value for timeout-minutes cannot be a numerical string.

"For numerical comparison, the fromJSON() function can be used to convert a string to a number"

https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions

Alright, sounds good! 👍

@albin-mullvad albin-mullvad force-pushed the add-repeat-arguments-for-self-hosted-e2e-tests-droid-1266 branch from 43a8cdb to 0b93056 Compare August 22, 2024 09:22
@albin-mullvad albin-mullvad merged commit 0fb19ae into main Aug 22, 2024
22 checks passed
@albin-mullvad albin-mullvad deleted the add-repeat-arguments-for-self-hosted-e2e-tests-droid-1266 branch August 22, 2024 09:26
@niklasberglund niklasberglund self-assigned this Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Android Issues related to Android
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants